Tracking system for visual effect triggering using induced magnetic field and predictive analysis

ABSTRACT

A system comprises active magnetic emitters positioned within an area, passive magnetic emitters configured to be moved within the area, a magnetic field detector configured to measure a strength and direction of a magnetic field within the area, and a processor in communication with the magnetic field detector. The passive magnetic emitters are configured to be integrated in, coupled to, or secured to at least one tracked object or tracked subject within the area. The processor is configured to evaluate at least one change in the measured strength and direction of the magnetic field end send a signal to a visual effect actuator or visual effects display to initiate a visual effect based on the at least one change. A method and computer program product relating to the system is also provided.

BACKGROUND

In today's technological environment, visual effects can provideentertainment. For example, physical effects, such as fireworks andexplosions can be engaging aspects of live shows and recorded content.In other examples, virtual content can be rendered and displayed inreal-time to provide desired visuals. As such, visual effects immerseviewers into a content creators world and deliver an exciting andengaging experience. However, triggering events should be timedcarefully to dive the proper effect, such as in the case of a fast-pacedaction sequence. Various systems and methods have been tested for use indetecting trigger events, but due to flaws inherent in such systems,including unacceptable degrees of latency, delay, inaccuracy, andunreliability in certain settings, they are unsuitable for achieving thelevel of realism desired.

Embodiments described herein address these problems and others,individually and, collectively.

SUMMARY

In one aspect, a system is provided. The system may comprise one or moreactive magnetic emitters configured to be situated at predeterminedpositions within a predefined area, one or more passive magneticemitters configured to be freely movable within the predefined area, amagnetic field detector configured to measure a strength and directionof a magnetic field within the predefined area, and a processor incommunication with the magnetic field detector. The one or more passivemagnetic emitters are configured to be integrated in, coupled to, orsecured to at least one tracked object or tracked subject within thepredefined area, and the processor is configured to evaluate at leastone change in the measured strength and direction of the magnetic fieldand to send a signal to a visual effect actuator or visual effectsdisplay to initiate the visual effect based on the at least oneevaluated change.

In another aspect, a method is provided. The method may comprisepositioning one or more active magnetic emitters at predeterminedpositions within a predefined area and introducing one or more passivemagnetic emitters into the predefined area. The one or more passivemagnetic emitters may be freely movable within the predefined area andconfigured to be integrated in, coupled to, or secured to et least onetracked object or tracked subject within the predefined area. The methodmay further comprise obtaining one or more measured values relating to astrength and direction of a magnetic field within the predefined area,evaluating at least one change relating to the measured strength anddirection of the magnetic field, and sending a signal to a visualeffects device to initiate the visual effect based on at least oneevaluated change in the magnetic field within the predefined area.

In yet another aspect, a computer program product comprises anon-transitory computer readable storage device having a computerreadable program stored thereon. The computer readable program whenexecuted on a computer causes the computer to obtain one or moremeasured values relating to a strength and direction of a magnetic fieldwithin a predefined area, evaluate at least one change in the measuredstrength and direction of the magnetic field, and send a signal to avisual effects device to initiate the visual effect based on the atleast evaluated change. The predefined area comprises one or more activemagnetic emitters positioned at predetermined locations within thepredefined area, and the predefined area further comprises one or morepassive magnetic emitters that are freely movable within the predefinedarea and are configured to be integrated in, coupled to, or secured toat least one tracked object or tracked subject within the predefinedarea.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned features of the present disclosure will become moreapparent with reference to the following description taken inconjunction with the accompanying drawings, wherein like referencenumerals denote like elements and in which:

FIG. 1 illustrates a block diagram of a system for initiating visualeffects according to embodiments.

FIG. 2 illustrates a block diagram of a processing computer for trackingand predicting pose of subjects/objects according to embodiments.

FIG. 3 illustrates a flow diagram of a method for initiating a visualeffect according to embodiments.

FIG. 4A illustrate an example of an area where visual effects can betriggered according to an embodiment.

FIG. 4B illustrates the example of the area of FIG. 4A after the visualeffects are triggered.

FIG. 5A illustrate an example of an area where visual effects can betriggered according to another embodiment.

FIG. 5B illustrates the example of the area of FIG. 5A after the visualeffects are triggered.

DETAILED DESCRIPTION

A system is provided for initiating visual effects. The systems andmethods described herein are well-suited for various visual effectssettings, notably in film production and live entertainment. In suchsettings, special considerations must be made as to the types oftracking methods used. In prior systems, tracked subjects/objectsrequired active magnetic elements (e.g., relying on powered emissionand/or time multiplexing) to be fitted and attached directly thereon,whereas embodiments herein integrate a passive magneticfield-interactive component in the tracked subject/object, with activeelements standing alone and offset in space from the trackedsubjects/objects. Because the emitter/detector pair is standalone,embodiments allow for the computation of a three-dimensional volume map(heat map) of the induced magnetic field. Embodiments not only allow forcomputation of a strength vector but also for the ability to makedifferential comparisons of the neat map as it changes, thereby allowinga spatial motion function to be modeled. This may allow for the abilityto perform predictive analysis and plan feedback to be triggered by thepredicted state, and not merely the actual state. The benefit providedis that physical or virtual effects that require a ramp-up time foractivation may be efficiently “spun-up” without user-detectible lag. Asdescribed herein, the terms “spun-up” and “ramp up” may refer to initialactions required for a visual effect to be executed. For example, for avisual effect actuator that relies on a mechanical, electromechanical,chemical, or other physical process to achieve a visual effect, the“spin-up time” or “ramp-up time” may be a period of time in which saidprocess is occurring, and thus creates a delay between the time acommand to initiate the visual effect is received and the time at whichthe visual effect is actually executed. For a visual effects display,the “ramp-up time” may be a period of time it may take the visualeffects display to render and display visual graphics as the desiredvisual effect, which may be dependent on the computations required forrendering the visual graphics and/or the characteristics of the hardwareused in performing said computations, such as the memory and processingpower available to the visual effects display.

As one example, in a film-production set implementation relating tophysical effects, a film-set camera operator may be shooting a wanderinghandheld shot following some action where a fire hydrant may explodewith a fountain of water just as the camera's field of view passes overthe hydrant. Using conventional approaches which operated withoutpredictive information, the camera operator would need to stop his orher previous motion and focus on the hydrant as a direction is given toactivate the water. Essentially, it would be clear to a sophisticatedviewer that the camera operator knew the water was coming, and thusdegrading the realism and immersion for the viewer. In contrast,embodiments herein may anticipate the camera's motion and may activatethe physical effect (e.g., the explosion of water) in a timely mannerthat preserves realism in the shot.

In a separate implementation for a virtual game or virtual reality (VR)interaction, the effect desired may be one of a number of possiblecompute-intensive display effects. The display effects may requirememory-hungry particle or physics computations that require startup timeto load into memory and then “pre-heat” an initial condition based onthe geometric and physics-engine state of the scene. If an effect wereto interact with the geometric shape of three-dimensional objects in thecurrent and upcoming frames of the scene, the predictive capability ofsystems herein may allow the software state to begin initialization andprepare for the predicted outcome. This may allow for a greater level ofvisual realism that may not be possible without the predictiveinformation of the software state provided by embodiments of theinvention.

Furthermore, conventional tracking systems rely on indirect simultaneouslocalization and mapping (SLAM), which may be doubly prone to error,once from the magnetic field error and then again from SLAM error. Assuch, the calculated position of a tracked subject/object may notcorrectly match the actual physical position of the trackedsubject/object in a precise manner. Additionally, other systems that usemagnetic tracking are prone to inaccuracy when tracking metallic objectsthat create magnetic interference. If the tracked device is a camera,prior systems required users to establish an offset of about a foot fromthe tracker to a camera body to achieve even moderate levels ofaccuracy. In systems and methods herein, instead of being inhibited bythe magnetic interference of the camera's metal components, embodimentsbenefit from additional magnetic elements present in the tracking area.

FIG. 1 illustrates a block diagram of a system for initiating visualeffects according embodiments. System 100 may comprise a visual effectsarea 110. Within or near the visual effects area 110, system 100 mayfurther comprise one or more active magnetic emitter(s) 111, one or moretracked subject(s) or object(s) 112, and one or more visual effectsdeice(s) 113. Furthermore, the system 100 may further comprise one ormore passive magnetic emitter(s) 112A that may be integrated in, coupledto, or secured to one or more tracked subjects/objects 112. For example,one or more passive magnetic emitter(s) 112A can be integrated orattached to various objects in a scene to track their movement andlocation. As another example, a tracked subject may be an individualplayer in a game, and one or more passive magnetic emitter(s) 112 may beintegrated into wearable devices (e.g. headsets, vests, wristbands,gloves, shoes, etc.) so as to identify the player's pose during gameplay(as described in greater detail below).

The visual effects area 110 may be an area where objects and/or subjectscan be tracked and/or visual effects may be initiated. For example, thevisual effects area 110 may be a production set (e.g., a physical areafor shooting/recording a scene for a movie or TV show), a live show set(e.g., a theme park attraction or stage show), or a gaming area (e.g,. aplay area/simulation area for a hyperreality, mixed reality, augmentedreality, and/or virtual reality game, or similar entertainmentexperience).

The tracked subject/object 112 may be any subject/object within thevisual effects area 110 whose position, orientation, pose, and/or othercharacteristic movement or gesture may affect initiation of a desiredvisual effect. For example, on a movie set, the tracked subject/object112 may be any one of a camera, camera equipment, lighting equipment(e.g., a spotlight, projector, or lighting fixture), prop, costume,actor/actress, vehicle, etc. As another example, on a live show set, thetracked subject/object 112 can also be a viewer of the live show. In yetanother example, in a gaming area, the tracked subject/object may be agame controller, headset or other wearable device, play objects in thegaming area, or even the players themselves. In these examples andothers, the position, pose, orientation of the tracked subject/objects,or component thereof may be tracked. For example, system 100 may trackwhere viewers are looking or where a camera's viewing frustum ispointed. As described herein, pose may refer to a characteristicposition or movement. As some non-limiting examples, a tracked subjectmay have a pose of “arms up and head to the side”, “crouching andlooking up”, “pointing in the northwest direction”, etc. The pose of atracked subject may be determined using a plurality of trackable objectsor devices worn by the tracked subject. For example, by tracking theposition and orientation of a combination of headset(s) and handcontroller(s) by a user, his or her pose may be determined.

The passive magnetic emitter(s) 112A may be magnetic elements thatnaturally produce a magnetic field without active power. For example,the passive magnetic emitter(s) 112A may comprise one or more permanentmagnets (e.g., made from ferromagnetic or ferrimagnetic materials). Inone embodiment, the passive magnetic emitter(s), may be a magnetic arrayor preconfigured arrangement of magnets, such as a Halbach arrangementof magnets (Halbach array) or other geometric configuration.

The visual effects device(s) 113 may be a device for executing a visualeffect. In various implementations, the visual effects device(s) 113 mayinclude visual effect actuators, visual effects displays, orcombinations thereof. For example, the visual effects device(s) 113 maybe any one of a lighting effects device, a display device, apyrotechnics device, and/or an electromechanical device, in variousembodiments. As such, a corresponding visual effect may be a lightingeffect, a rendering of virtual content, a pyrotechnic effect (e.g.,explosions, fireworks, flames, etc.), or some other controlled effect(e.g., activation of a machine, prop, sound emitting element, or otherdevice) provided by the aforementioned visual effects device(s) 113.

Activation of the visual effects device(s) 113 and/or initiation of acorresponding visual effect, may be controlled/executed by theprocessing computer 120. The processing computer 120 may be, forexample, processing computer 200 of FIG. 2 (described in greater detailfurther below in this description). According to embodiments, theprocessing computer 120 may initiate the corresponding visual effectbased on one or more magnetic field measurement(s) 150 that are measuredby a magnetic field detector 130 and communicated to the processingcomputer 120.

The magnetic field detector 130 may be a device for taking magneticfield measurements. In embodiments, the magnetic field detector 130 maybe a magnetometer for measuring a direction, strength, and/or relativechange of a magnetic field induced in the visual effects area 110 (i.e.induced magnetic field 140), such as a vector magnetometer or scalarmagnetometer. For example, the magnetic field detector 130 may be anyone of a proton magnetometer, Overhauser effect magnetometer, cesiumvapor magnetometer, potassium vapor magnetometer, rotating coilmagnetometer, hail effect magnetometer, fluxgate magnetometer, SQUIDmagnetometer, spin-exchange relaxation-free (SERF) magnetometer, and/ormagneto-resistive device. In some embodiments, the magnetic fielddetector 130 may be stationary, or in other embodiments, may beconfigurable to move (e.g., translate and/or rotate).

The magnetic field detector 130 includes one or more magnetic sensor(s)130A. In some embodiments, an induced magnetic field 140 present in thevisual effects area 110 at various points in time may be sensed by themagnetic sensor(s) 130A of the magnetic field detector 130. The magneticsensor(s) 130A may be an integrated circuit for detecting a magneticfield based on an evaluation of a physical effect, such as a change involtage or resonant frequency (Lorentz Force) sensed electronically or amechanical displacement sensed optically. For example, the magneticsensor(s) 130A may comprise a microelectromechanical systems (MEMS)magnetic field sensor.

In some embodiments, magnetic field measurements(s) 150 measured bymagnetic field detector 130 may be communicated to the processingcomputer 120. The processing computer 120 may be programmed to evaluatethe magnetic field measurement(s) 150 and compare them againstpredefined visual effects initiation criteria. The predefined visualeffects initiation criteria may comprise a desired or expected pose ofthe tracked subject/object 112 in the pre-established, predefined visualeffects area 110. For example, the visual effects device(s) 113 may bean explosive device, which may be activated once a particular cameraangle or desired pose for a shot has been achieved and that allows thecamera operator to record a triggered explosion as it is happening. Inanother example, the visual effect may be a display of virtual contentin a hyperreality game, and the visual effects initiation criteria maybe a detected head pose of a user in the game where the head pose issuch that the user's attention is directed at the intended location ofwhere the virtual content should be placed in the gaming environment.

An effect initiation signal 160 may be sent by the processing computer120 to the visual effects device(s) 113 to initiate the visual effect inwithin a predetermined amount of time once the processing computer 120has determined that one or more visual effects initiation criteria hasbeen met. For example, the effect initiation signal 160 may comprise acommand to activate the visual effects device 113 and/or command thevisual effects device 113 to begin the desired visual effect. The effectinitiation signal 160 may be executed by a processor of processingcomputer 120 based on at least the magnetic field measurement(s) 150received from the magnetic field detector 130. As previously mentioned,the effect initiation signal 160 may include an activation signal toactivate or “ramp up” the visual effects device(s) 113. Furthermore, theeffect initiation signal 160 may occur based on a predicted state of theinduced magnetic field 140. According to embodiments, processingcomputer 120 may be configured to generate and update one or more heatmaps of the induced magnetic field, which may be compared against apredictive model. For example, the magnitude and direction of themagnetic field can be mapped spatially. In one implementation, the oneor more heat maps can be visually expressed in a manner similar to atemperature heat map, where the strength of a magnetic field may beexpressed using a color scale (e.g. bluer hues representing lowermagnitude areas, redder hues representing high magnitude areas). Fromthe comparison, the processing computer 120 may predict a next update tothe heat map(s), which may allow for the effect initiation signal 160 tobe sent to the visual effects device 113 prior to a desired pose of thetracked pose of the tracked subject/object 112 being actually achieved.For example, a comparison of the heat map(s) to the predictive model mayallow the processing computer 120 to determine the trackedsubject/object 112's movement and predict that the trackedsubject/object 112 will achieve the desired pose specified in the visualeffects initiation criteria at the next time step or at least at a latertime step for which a prediction can be made. Thus, system 100 may beused to track and predict pose, movement, orientation, and position of atracked subject/object 112 so that a visual effect may be timelytriggered.

FIG. 2 illustrates a block diagram of a processing computer for trackingand predicting pose of subjects/objects according to embodiments.Processing computer 200 may be a computer for performing processingfunctions relating to tracking and prediction of trackedsubject/objects, evaluating visual effects initiation criteria, andsignaling initiation of a visual effect, amongst other functions. Theprocessing computer 200 may be, for example, processing computer 120 ofFIG. 1 , and may thus share the same or similar features and integrationinto system 100 of FIG. 1 . Similarly, any mentioning of visual effectsareas, tracked subjects/objects, active magnetic emitters, passivemagnetic emitters, induced magnetic fields, visual effects devices,magnetic field detectors, magnetic sensors, magnetic field measurements,and/or effect initiation signaling used in describing FIG. 2 may applyto the corresponding elements in FIG. 1 and vice versa.

As shown in FIG. 2 , the processing computer 200 may comprise one ormore processor(s) 210, one or more I/O interface(s) 220, one or morenetwork interface(s) 230, and at least one computer-readable medium 240.Processor(s) 210 may comprise one or more processors for performingcalculations and executing computer-readable instructions. For example,processor(s) 210 may comprise one or more central processing units(CPUs) and/or one or more graphics processing units (GPUs). The I/Ointerface(s) 220 may comprise hardware for facilitating communicationbetween one or more peripheral input/output devices and the processor(s)210. For example, each of the I/O interface(s) 220 may include acommunication bus and processing logic necessary for interpreting deviceaddresses. The network interface(s) 230 may comprise one or morecommunication interfaces for communicating over a network. For example,the network interface(s) 230 may comprise a wired or wireless connectionfor transmitting, receiving, formatting, and reformatting messages overa standard communication protocol, such as over WiFi, Bluetooth nearfield Communication (NFC), RFID, USB, local area networks, cellularnetworks (e.g., 3G, 4G, 5G, LTE, etc.), global networks (e.g., theinternet), etc. The computer-readable medium 240 comprise one or morenon-transitory computer-readable mediums for storing instructionsexecutable by the processor(s) 210. In embodiments, the instructionsexecutable by the processor(s) 210 may be in the form ofcomputer-readable code (i.e. computer code or program code). Thecomputer-readable medium 240 may take the form of any number of internalor external memory devices, such as random-access memo (RAM), dynamicrandom-access memory (DRAM), static random-access memory (SRAM), flashmemory, read-only memory (ROM), programmable read-only memory (PROM),erasable programmable read-only memory (EPROM), electrically erasableprogrammable read-only memory (EEPROM), to a few non-limiting examples.In certain embodiments, the processor-executable instructions may bestored in the computer-readable medium 240 as modules or portions ofcode. The modules may comprise measurement module 240A, measurementevaluation module 240B, visual effects signaling module 240C, mappingand update module 240E, prediction module 240G, and criterion comparisonmodule 240I. Additionally computer-readable medium 240 may comprise setsof data or other stored information, such as map(s) 240D, predictivemodel(s) 240F, and visual effects initiation criteria 240H.

Measurement module 240A may comprise code for retrieving and/or takingmeasurements, notably magnetic field measurements received from amagnetic field detector. For example, measurement module 240A maycomprise instructions for controlling a measurement function of themagnetic field detector and/or retrieving magnetic field measurementsfrom the magnetic field detector at various points in time. Measurementevaluation module 240B may comprise code for evaluating one or moremeasurements, such as magnetic field measurements or one or more othervalues relating to a magnetic field within a predefined area (i.e.visual effects area). The one or more measurements or values relating tothe magnetic field may directly or indirectly measure the completemagnetic field or may directly or indirectly measure vector componentsof the magnetic field. For example, the one or more measurements orvalues relating to the magnetic field may include a strength of themagnetic field, a direction of the magnetic field, a magnetic flux, or acombination thereof. Evaluating the one or more measurements maycomprise evaluating at least one change in a magnetic field within thevisual effects area 110. In one embodiment, evaluating the one or moremeasurements may comprise using magnetic field measurements to update aheat map of an induced magnetic field relative to a spatial mapping ofthe visual effects area 110 at various points in time (i.e. time steps).In embodiments, an update made to the heat map at a first time step maybe used to predict a next update to the heat map for a subsequent timestep, prior to new magnetic field measurements being obtained. As such,evaluating the one or more measurements may further comprise comparingthe updated heat map to a predictive model and predicting a next updateto the heat map for a second time step based on the comparison. Theupdating, comparing, and predicting steps may be initiated using code ofthe measurement evaluation module 240B and may be executed using code ofmapping and update module 240E, prediction module 240G, and criterioncomparison module 240I respectively. Visual effects signaling module240C may comprise code for communicating signals for executing one ormore visual effects. In embodiments, visual effects signaling module240C may comprise instructions for sending a signal to a visual effectsdevice to initiate a visual effect. This signal may be based on at leastone evaluation made using instructions of the measurement evaluationmodule 240B, such as based on an updated heat map or based on apredicted next update.

Map(s) 240D may comprise one or maps relating to at least one predefinedarea, such as relating to visual effects area 110. In embodiments, themap(s) 240D may comprise a plurality of volumetric heat maps generatedand updated based on magnetic field measurements. The plurality ofvolumetric heat maps may each represent magnetic field vectors (strengthand direction) against a 3D spatial mapping of the predefined area. Forexample, the heat map may be a visual map in which spatial areas withgreater magnetic field strength may be portrayed in a different colorthan lower magnetic field strength areas (e.g. red vs blue or darker vslighter colors), and may further portray arrows for the direction inwhich the magnetic field appears to be moving in time. Furthermore,map(s) 240D may comprise the 3D spatial mapping of the predefined area.

Mapping and update module 240E may comprise code for obtaining,generating, and/or updating the map(s) 240D. In embodiments, updatingthe map(s) 240D may comprise updating a heat map at each time step(e.g., during the course of a live show, recorded production, or livegameplay taking place within the visual effects area). As previouslymentioned, the heat map may be a map of a measured magnetic fieldrelative to a spatial mapping of the visual effects area 110.Furthermore, mapping and update module 240E may comprise instructionsfor generating the map(s) 240D “on the fly”, such as code forcalculating and displaying the map(s) 240D at each time step based onsensor data obtained in real-time.

Predictive model(s) 240F may comprise one or more predictive model(s).In embodiments, the predictive model(s) may include at least onepredictive model relating to the movement of tracked subjects/objectswithin a predefined area. This may include a model for predicting a nextupdate to map(s) 240D based on data obtained at previous points in time(e.g. based on previous updates to map(s) 240D). For example, processingcomputer 200 may update a heat map of an induced magnetic field over thecourse of 5 different time steps, and the predictive model may comprisea function, trend, or, statistical distribution used to predict asubsequent update to the heat map based on the updates made at thoseprevious 5 time steps. Thus, the predictive model may give a real-timeprediction of pose, orientation, and motion of tracked subjects/objects,based on the modeled behavior of a magnetic field induced in the visualeffects area. Prediction module 240G may comprise instructions formaking predictions. This may include code for comparing an updated heatmap to predictive model(s) 240F and to make a prediction as to asubsequent update to the heat map based on the comparison. For example,prediction module 240G may comprise instructions for applying magneticfield data to the predictive model(s) 240F and for calculating andinterpolating future updates to the heat map. In other words, theprediction module 240G may comprise code for predicting movement oftracked subjects/objects in an induced magnetic field based on mappedtrends and modeled behavior of the magnetic field.

Visual effects initiation criteria 240H may comprise criteria forinitiating a visual effect. In embodiments, the visual effectsinitiation criteria may comprise a determined pose, position,orientation, and/or movement of a tracked subject/object present withina predefined area (e.g. within visual effects area 110). For example,one criterion may be a specified direction that a tracked subject shouldbe facing prior to the effect, such as directly facing the portion ofthe visual effects area where the desired visual effect is localized.Criterion comparison module 240I may comprise code for comparing data toat least one criterion specified in visual effects initiation criteria240H. For example, the data may include data about trackedsubjects/objects in a visual effects area obtained in real-time. Inembodiments, comparing the data to the at least one criterion maycomprise comparing a predicted next update for a heat map of an inducedmagnetic field to the visual effects initiation criteria. This maycomprise instructions for relating the magnetic field given by the heatmap after the predicted next update to a mapped pose, position,orientation, and/or movement of magnetic emitters that correspond totracked subjects/objects. For example, the criterion comparison module240I may comprise code for determining that the induced magnetic fieldat the next time step will be indicative of a tracked subject turningtowards the portion of the visual effects area where the visual effectwill be localized once initiated. Thus, processing computer 120 maydetermine that a signal should be sent to initiate the visual effect ina timely manner, such as just before the desired pose of the trackedsubject/object is achieved or at a point in time that accounts for theramp-up time of the visual effects device (e.g. effect takes one secondto ramp up, initiate one second before pose is predicted to occur).

FIG. 3 illustrates a flow diagram of a method for initiating a visualeffect according to embodiments. In embodiments, method 300 shown may beimplemented by a processing computer, such as processing computer 200 ofFIG. 2 and/or processing computer 120 of system 100 shown in FIG. 1 .Similarly, visual effects area(s), magnetic emitter(s), induced magneticfield(s), magnetic field measurement(s), and magnetic fielddetector(s)/sensor(s) mentioned in the description of method 300 mayrelate to corresponding components of system 100, as shown by FIG. 1 .

Furthermore, the method may be a computer-implemented method, andprocessing instructions for performing the method may be a computerprogram product stored in a non-transitory computer-readable storagedevice, such as computer-readable medium 240 of processing computer 200,for example. As such, when executed on the processing computer 200, thecomputer program product may cause the processing computer to performthe method.

Looking at FIG. 3 , in step S301, the processing computer obtains one ormore measured values relating to a magnetic field within a predefinedarea (e.g., the visual effects area). In embodiments, the predefinedarea may comprise active magnetic emitter(s) situated at predeterminedlocations within the predefined area and passive magnetic emitter(s)freely movable within the predefined area. For example, the activemagnetic emitter(s) may be actively powered magnetic emission devicespositioned at stationary locations at, near, or around a visual effectsarea so that a measurable magnetic field may be induced. In addition,passive magnetic emitter(s) (e.g., Halbach arrays or other permanentmagnet arrangement) may be integrated into, secured to, or otherwisecoupled to a tracked subject or object moving within the visual effectsarea, thereby interacting with the induced magnetic field.

In step S302, the processing computer may evaluate the one or moremeasured values relating to the magnetic field. In embodiments,evaluating the one or more measured values relating to the magneticfield may comprise obtaining a heatmap of the magnetic field relative toa spatial mapping of a predefined area and updating the heat map of themagnetic field using the one or more measured values at a first timestep. Furthermore, at step S302, the processing computer may perform thefunctions of comparing the updated heat map to a predictive model andpredict, based on the comparison, a next update of the heat map for asecond time step.

In step S303 the processing computer may send a signal to a visualeffects device to initiate the visual effect based on at least oneevaluated change in the magnetic field within the predefined area. Inembodiments, at step S303, the processing computer may further performthe functions of comparing a predicted next update of the heatmap of themagnetic field for the second time step to a visual effects initiationcriteria. Based on the comparison, the processing computer may send asignal to the visual effects device to initiate the visual effect at thesecond time step based on the comparison. As such, the fulfillment ofvisual effect initiation criteria may be anticipated for a second timestep based on magnetic field measurements obtained at a first time step.Upon anticipating fulfillment of the visual effect initiation criteria,the signal to the visual effects device may be sent and the visualeffect activated in the visual effects area.

FIG. 4A and FIG. 4B illustrate an example of an area where visualeffects can be triggered according to an embodiment. As shown, apredefined visual effects area 410 is illustrated in two different timeframes: time frame 401 (time=t) and time frame 402 (time=t+1). Visualeffects area 410 is depicted as a film production set and the trackedobject 412 is depicted as a camera. Embedded in or attached to thecamera 412 may be a passive magnetic emitter 412A, such as a Halbachemitter. Looking at the time frame 401, a camera operator is holdingcamera 412 and pointing it at elements in the scene during a shot takingplace in visual effects area 410. Also in the visual effects area 410,may be one or more active magnetic emitters 411 for inducing a magneticfield, and a magnetic sensor 430A that detects and obtains informationabout the induced magnetic field. As the camera operator moves aroundthe visual effects area 410 and orients the camera 412 in differentpositions and directions, its pose affects the state of the inducedmagnetic field by way of the passive magnetic emitter 412A embeddedtherein/thereon. Thus, one or more measured values obtained usingmagnetic sensor 430A may provide information that can be correlated tothe camera's pose and motion (e.g., by way of a processing computer). Intime frame 401, the processing computer (not shown) may determine thatthe camera's motion is indicative of the camera achieving the desiredpose for a visual effect in the next time step. For example, theprocessing computer 200 of FIG. 2 may generate a heat map and predictthat the camera's viewing frustum will soon point at a portion of thevisual effects area where a visual effect is localized. Therefore, theprocessing computer may send a signal to a visual effects device 413 toramp up and initiate the visual effect in the time frame 402.

In the example of FIG. 4A, a plurality of active magnetic emitters 411are illustrated in a clustered arrangement. However, in other examplesin which a plurality of active magnetic emitters 411 are provided, theactive magnetic emitters 411 may be positioned throughout the visualeffects area 410, for example, in predetermined positions surroundingthe passive magnetic emitter 412A, the visual effects device 413, orboth of the passive magnetic emitter 412A and the visual effects device413.

FIG. 5A and FIG. 5B illustrate an example of an area where visualeffects can be triggered according to another embodiment. As shown, apredefined visual affects area 510 is illustrated in two different timeframes: time frame 501 (time=t) and time frame 502 (time=t+1). Visualeffects area 510 is depicted as a theme park attraction, such as anamusement ride comprising a ride vehicle navigating through various showelements provided in the visual effects area 510. In the ride vehiclemay be a plurality of riders/amusement park guests that serve as thetracked subjects in the system, each wearing a passive magnetic emitter512A. For example, passive magnetic emitters 512A may be implemented as,or integrated into, hats or other headgear that can be worn.

Looking at the time frame 501, the guests 512 may be looking around atthe different show elements in the scene as the ride vehicle passesthrough the visual effects area 510. Also in the visual effects area510, may be one or more active magnetic emitters 511 for inducing amagnetic field, and a magnetic sensor 530A that detects and obtainsinformation about the induced magnetic field. As the guests look indifferent directions, their head pose affects the state of the inducedmagnetic field by way of the passive magnetic emitters 512A. Thus, oneor more measured values obtained using magnetic sensor 530A may provideinformation that can be correlated to the head pose and motion of theguests (e.g by way of a processing computer). In time frame 501, theprocessing computer (not shown) may determine that guest head motion isindicative of achieving the desired pose for a visual effect (i.e., thevisual effects initiation criterion) in the next time step. For example,the processing computer 200 of FIG. 2 may generate a heat map andpredict that the majority, or at least a predetermined number, of guestsin the ride vehicle will soon look directly at a portion of the visualeffects area where a visual effect is localized. Therefore, theprocessing computer may send a signal to a visual effects device 513 toramp up and initiate the visual effect in the time frame 502.

In the example of FIG. 5A, a plurality of active magnetic emitters 511are illustrated in a clustered arrangement. However, in other examplesin which a plurality of active magnetic emitters 511 are provided, theactive magnetic emitters 511 may be positioned throughout the visualeffects area 510, for example, in predetermined positions surroundingthe passive magnetic emitter 512A, the visual effects device 513, orboth of the passive magnetic emitter 512A and the visual effects device513.

The processes described herein may be implemented in a specializedprocessor. Such a processor will execute instructions, either at theassembly, compiled or machine-level, to perform the processes. Thoseinstructions can be written by one of ordinary skill in the artfollowing the description of the figures corresponding to the processesand stored or transmitted on a computer readable medium. Theinstructions may also be created using source code or any other knowncomputer-aided design tool. A computer readable medium may be anymedium, e.g., computer readable storage device, capable of carryingthose instructions and include a CD-ROM, DVD, magnetic or other opticaldisc, tape, silicon memory (e.g., removable, non-removable, volatile ornon-volatile), packetized or non-packetized data through wireline orwireless transmissions locally or remotely through a network. A computeris herein intended to include any device that has a specialized,general, multi-purpose, or single purpose processor as described above.For example, a computer may be a desktop computer, laptop, smartphone,tablet device, set top box, etc.

It is understood that the apparatuses, systems, computer programproducts, and processes described herein may also be applied in othertypes of apparatuses, systems, computer program products, and processes.Those skilled in the art will appreciate that the various adaptationsand modifications of the aspects of the apparatuses, systems, computerprogram products, and processes described herein may be configuredwithout departing from the scope and spirit of the present apparatuses,systems, computer program products, and processes. Therefore, it is tobe understood that, within the scope of the appended claims, the presentapparatuses, systems, computer program products, and processes may bepracticed other than as specifically described herein.

What is claimed is:
 1. A system for initiating a visual effect, thesystem comprising: one or more active magnetic emitters positionedwithin an area; one or more passive magnetic emitters configured to bemovable within the area; a magnetic field detector configured to measurea strength and direction of a magnetic field within the area; and aprocessor in communication with the magnetic field detector, wherein theone or more passive magnetic emitters are configured to be integrated incoupled to, or secured to at least one tracked object or tracked subjectwithin the area, and wherein the processor is configured to evaluate atleast one change in the measured strength and direction of the magneticfield and to send a signal to a visual effect actuator or visual effectsdisplay to initiate the visual effect based on the at least one change.2. The system of claim 1, wherein the visual effect comprises a lightingeffect, a display of virtual content, or a pyrotechnic effect.
 3. Thesystem of claim 1, wherein the processor is further configured to:generate a heat map of the magnetic field relative to a spatial mappingof the area; generate at least one update of the heat map, the at leastone update being generated using the measured strength and direction ofthe magnetic field taken during a previous time step; apply the at leastone update to generate an updated heat map; compare the updated heat mapto a predictive model stored in the processor; and predict, based on thecomparison, at a current time step, a next update of the heat map for asubsequent time step.
 4. The system of claim 3, wherein the processor isfurther configured to: compare the predicted next update of the heat mapfor the subsequent time step to a visual effects initiation criterion;and during the current time step, send the signal to the visual effectactuator or to the visual effects display to initiate the visual effectin response to the visual effects initiation criterion being satisfiedsuch that the visual effect is executed at the subsequent time step. 5.The system of claim 4, wherein the visual effects initiation criterioncomprises a predetermined position, orientation, or pose of the at leastone tracked object or of the tracked subject in the area.
 6. The systemof claim 1, wherein the area is one or more of: a production set, liveshow set, or a gaming area.
 7. The system of claim 1, wherein thetracked object comprises a camera, a prop, a costume, a wearable device,or a combination thereof.
 8. The system of claim 1, wherein the one ormore passive magnetic emitters comprise a Halbach array.
 9. A method ofinitiating a visual effect, the method comprising: positioning one ormore active magnetic emitters within an area; introducing one or morepassive magnetic emitters into the area, the one or more passivemagnetic emitters being movable within the area and configured to beintegrated in, coupled to, or secured to at least one tracked object ortracked subject within the area; obtaining one or more measurementsrelating to a strength and direction of a magnetic field within thearea; evaluating at least one change in the one or more measurementsrelating to the strength and direction of the magnetic field; andsending a signal to a visual effects device to initiate the visualeffect based on at least one change in the one or more measurementsrelating to the magnetic field within the area.
 10. The method of claim9, further comprising: generating a heat map of the magnetic fieldrelative to a spatial mapping of the area.
 11. The method of claim 10,further comprising: generating at least one update of the heat map, theat least one update being generated using the one or more measurementsrelating to the strength and direction of the magnetic field takenduring the previous time step; and applying the at least one update togenerate an updated heat map.
 12. The method of claim 11, furthercomprising: comparing the updated heat map to a predictive model; andpredicting, based on the comparison, at a current time step, a nextupdate of the heat, map for a subsequent time step.
 13. The method ofclaim 12, further comprising: comparing the predicted next update of theheat map for the subsequent time step to a visual effects initiationcriterion; and during the current time step, sending the signal to thevisual effects device to initiate the visual effect in response to thevisual effects initiation criterion being satisfied such that the visualeffect is executed at the subsequent time step.
 14. The method of claim13, wherein the visual effects initiation criterion comprises apredetermined position, orientation, or pose of the at least one trackedobject or of the tracked subject in the area.
 15. The method f claim 9,wherein the one or more passive magnetic emitters comprise a Halbacharray.
 16. A computer program product comprising a non-transitorycomputer readable storage device having a computer readable programstored thereon, wherein the computer readable program when executed on acomputer causes the computer to: obtain one or more measurementsrelating to a strength and direction of a magnetic field within an area;evaluate at least one change in the one or more measurements relating tothe strength and direction of the magnetic field; and send a signal to avisual effects device to initiate the visual effect based on the atleast one change, wherein one or more active magnetic emitters arepositioned within the area, and wherein one or more passive magneticemitters are movable within the area and configured to be integrated in,coupled to, or secured to at least one tracked object or tracked subjectwithin the area.
 17. The computer program product of claim 16, whereinthe computer readable program when executed on the computer furthercauses the computer to: generate a heat map of the magnetic fieldrelative to a spatial mapping of the area; generate at least one updateof the heat map, the at least one update being generated using themeasured strength and direction of the magnetic field taken during aprevious time step; apply the one or more updates to generate an updatedheat map; compare the updated heat map to a predictive model stored in aprocessor of the computer; and predict, based on the comparison, at acurrent time step, a next update of the heat map for a subsequent, timestep.
 18. The computer program product of claim 17, wherein the computerreadable program when executed on the computer further causes thecomputer to: compare the predicted next update of the heat map for thesubsequent time step to a visual effects initiation criterion; andduring the current time step, send the signal to the visual effectsdevice to initiate the visual effect responsive to the visual effectsinitiation criterion being satisfied such that the visual effect isexecuted at the subsequent time step.
 19. The computer program productof claim 18, wherein the visual effects initiation criterion comprises apredetermined position, orientation, or pose of the at least one trackedobject or of the tracked subject in the area.
 20. The computer programproduct of claim 16, wherein the one or passive magnetic emitterscomprise a Halbach array.